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ABSTRACT 


The  Department  of  Defense  (DOD)  spends  billions  acquiring  weapons  systems,  supplies, 
and  services.  The  contract  management  process  has  to  be  executed  diligently  to  ensure 
the  government  is  receiving  the  highest  return  on  investment.  The  process  has  six  steps, 
two  of  which  relate  to  the  source  selection  strategy:  solicitation  planning  and  source 
selection.  Once  the  acquisition  team  determines  whether  to  use  a  lowest  price  technically 
acceptable  (LPT A)  or  Tradeoff  source  selection  strategy,  they  evaluate  proposals  to 
determine  which  offer  presents  the  best  value  to  the  government. 

The  purpose  of  this  research  is  to  explore  potential  relationships  between  the 
source  selection  strategy  (LPTA  or  Tradeoff)  and  resultant  contract  outcomes.  This 
research  uses  data  collected  from  contract  files  and  related  documentation  from  two 
major  systems  commands  (Naval  Air  Systems  Command  and  Naval  Sea  Systems 
Command)  to  show  the  implication  of  the  LPTA  and  Tradeoff  source  selection  strategies. 
The  findings  suggest  that  an  LPTA  source  selection  strategy  has  a  significantly  shorter 
lead-time  to  contract  award.  The  findings  should  be  viewed  with  caution,  however,  as  the 
sample  size  consisted  of  only  six  LPTA  contracts.  This  report  concludes  with  two 
recommendations  to  improve  further  research  on  choosing  a  source  selection  strategy  and 
contract  outcomes. 
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I.  INTRODUCTION 


With  an  annual  budget  approaching  $600  billion,  the  Department  of  the  Defense’s 
(DOD)  spending  power  is  greater  than  the  gross  domestic  product  of  many  nations.  The 
Government  Accountability  Office  reported  that  the  DOD  spent  approximately  $310 
billion  acquiring  major  weapons  systems,  supplies,  and  services  in  fiscal  year  2013 
(GAO,  2014).  Government  acquisition  professionals  use  the  contract  management 
process  as  a  road  map  to  navigate  the  defense  acquisition  life  cycle  (Rendon  &  Snider, 
2008,  165).  The  contract  management  process  consists  of  pre-award,  award  and  post¬ 
award  phases.  The  pre-award  phase  includes  procurement  planning  and  solicitation 
planning;  the  award  phase  consists  of  the  source  selection,  in  which  the  contract  is 
awarded;  and,  finally,  the  post-award  phase  consists  of  contract  administration  and 
contract  closeout. 

A  critical  step  in  any  acquisition  program  is  choosing  a  contracting  award  strategy 
that  will  yield  the  highest  benefit  to  the  government.  The  two  primary  source  selection 
strategies  used  by  government  acquisition  professionals  to  detennine  the  proposal  that 
represents  the  best  value  to  the  government  are  lowest  price  technically  acceptable 
(LPTA)  and  Tradeoff  (also  known  as  full  Tradeoff  [FTO]).  While  there  are  several 
strategies  that  can  be  used,  the  basic  premise  underlying  all  of  these  strategies  is  the 
relative  importance  of  cost/price  and  non-cost/non-price  factors  (e.g.,  factors  related  to 
the  technical  capabilities,  managerial  capabilities,  past  performance,  etc.,  of  a  proposal.) 
The  Federal  Acquisition  Regulation  (FAR)  states  that  LPTA  is  “appropriate  for  use  when 
best  value  is  expected  to  result  from  selection  of  the  technically  acceptable  proposal  with 
the  lowest  evaluated  price.”  In  this  case,  cost/price  is  the  most  important  factor.  Tradeoff, 
on  the  other  hand,  must  be  used  when  best  value  can  be  obtained  by  choosing  the  “other 
than  the  lowest  priced  offeror  or  other  than  the  highest  technically  rated  offeror”  (FAR, 
2014,  Part  15.101-1).  In  this  case,  cost/price  may  be  less  important  than  other  non- 
cost/non-price  factors,  and  cost/price  can  be  “traded  off”  for  factors  deemed  more 
important  for  contract  success. 
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A. 


PURPOSE 


The  purpose  of  this  research  is  to  determine  potential  relationships  between  the 
source  selection  strategy  (LPT A  or  Tradeoff)  and  resultant  contract  outcomes.  The 
researchers  seek  to  determine  if  relationships  exist  between  the  source  selection  strategy 
and  various  contract  outcomes,  such  as  procurement  administrative  lead  time  (PALT), 
cost  and  schedule  overrun  or  underruns,  contractor  perfonnance  ratings,  and  other 
factors.  The  results  of  this  analysis  will  be  used  to  inform  the  DOD  contracting 
community  and  their  customers  of  the  potential  costs  and  benefits  associated  with 
choosing  a  particular  source  selection  strategy.  We  hope  to  provide  guidance  that  will 
help  acquisition  professionals  choose  the  source  selection  strategy  that  is  most 
appropriate  for  their  requirement. 

B.  RESEARCH  QUESTIONS 

This  research  intends  to  answer  the  following  questions: 

1 .  How  does  the  source  selection  strategy  affect  pre-award  metrics  (e.g., 
PALT,  number  of  solicitation  amendments,  number  of  protests)? 

2.  How  does  the  source  selection  strategy  affect  post-award  outcomes  (e.g., 
Contractor  Performance  Assessment  Reporting  Systems  [CPARS]  ratings, 
Earned  Value  Management  [EVM]  performance  metrics)? 

3.  Does  one  source  selection  strategy  consistently  fare  better  than  the  other  in 
tenns  of  both  pre-award  metrics  and  post-award  outcomes? 

4.  Does  the  contract  outcomes  (e.g.,  past  performance  data)  justify  the 
government  paying  a  premium  to  award  to  other  than  the  lowest  bidder? 

C.  METHODOLOGY 

The  research  methodology  consists  of  a  literature  review,  data  collection,  data 
analysis,  and  detennination  of  the  findings.  We  analyzed  the  source  selection  process  to 
include  the  best  value  continuum  and  the  associated  source  selection  strategies  (LPTA  or 
Tradeoff)  used  to  obtain  the  best  value.  We  reviewed  completed  contracts  and  contract- 
related  documentation  from  Naval  Air  System  Command  (NAVAIR)  and  Naval  Sea 
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System  Command  (NAVSEA).  The  team  also  collected  data  from  completed  contract 
files,  paying  particular  attention  to  the  source  selection  plans  and  solicitations  to 
detennine  which  contracts  were  awarded  based  on  an  LPTA  strategy  and  which  were 
awarded  based  on  a  Tradeoff  strategy.  In  the  cases  for  which  Tradeoff  was  chosen,  the 
research  team  examined  the  award  criteria  to  identify  which  criteria  were  most  important 
in  the  source  selection  strategy.  Commonly  used  evaluation  criteria  include,  but  are  not 
limited  to,  technical,  management,  past  perfonnance,  and  cost.  The  data  obtained  from 
the  contract  files  is  used  to  detennine  potential  relationships  between  the  source  selection 
strategy  (LPTA  or  Tradeoff)  and  resultant  contract  outcomes.  The  data  was  analyzed 
using  analysis  of  covariance  (ANCOVA)  procedures,  which  is  used  to  determine  if 
significant  differences  in  contract  outcomes  exist  between  the  two  award  strategies. 

D.  BENEFITS  OF  THE  RESEARCH 

The  purpose  of  this  research  is  to  detennine  potential  relationships  between  the 
source  selection  strategy  (LPTA  or  Tradeoff)  and  resultant  contract  outcomes.  The 
benefits  of  this  research  will  be  to  assist  federal  contracting  agencies  in  planning  there 
procurements  and  knowing  the  implications  of  a  LPTA  or  Tradeoff  source  selection 
strategy  in  terms  of  the  potential  contract  outcomes.  If  a  pattern  of  contract  outcomes  is 
identified,  this  research  could  help  naval,  DOD,  and  federal  acquisition  professionals 
choose  the  strategy  with  the  best  chance  of  producing  positive  contracting  outcomes,  thus 
potentially  saving  time  and  money  without  reducing  perfonnance  quality.  For  cases  in 
which  the  Tradeoff  process  is  used,  this  research  may  help  assesses  whether  or  not  the 
contract  outcomes  justified  paying  a  premium  cost/price. 

E.  LIMITATIONS  OF  THE  RESEARCH 

The  primary  limitation  of  the  research  is  the  size  of  the  statistical  sample.  Due  to 
time  constraints  and  the  size  of  the  research  team,  only  36  contracts  were  reviewed.  The 
time  allotted  to  review  contract  files  proved  to  be  significantly  shorter  than  what  was 
needed  to  gather  a  larger  sample  size.  Larger,  more  complex  procurements  naturally  took 
longer  to  review  and  locate  pertinent  data.  Another  major  challenge  the  team  encountered 
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while  collecting  data  was  the  lack  of  commonality  in  the  contract  files.  Though  each 
command  required  contracting  professionals  to  utilize  a  contract  file  checklist,  the  team 
noticed  a  variety  of  interpretations  and  utilizations  of  the  checklist.  Some  contract  files 
were  extremely  organized  and  revealed  that  the  contracting  professional  strictly  adhered 
to  the  checklist,  while  others  seemed  to  include  only  what  the  contracting  professional 
deemed  to  be  critical  items.  Consequently,  the  team  spent  a  significant  amount  of  time 
trying  to  locate  the  pertinent  data,  which  resulted  in  a  smaller  sample  size  than 
anticipated.  We  still  feel  the  data  is  worthy  of  analysis,  however  our  conclusions  should 
be  interpreted  with  caution  given  the  small  sample  size. 

F.  ORGANIZATION  OF  THE  REPORT 

This  report  is  organized  into  five  chapters,  including  this  introduction.  The  next 
chapter,  Chapter  II,  includes  a  review  of  the  literature  related  to  the  contract  management 
process,  and  source  selection  strategies.  Chapter  III  provides  an  overview  of  NAVAIR 
and  NAVSEA  and  their  contract  management  strategies.  Chapter  IV  presents  the  results 
and  findings  of  the  data  collected  and  answer  the  research  questions.  Finally,  Chapter  V 
provides  the  summary,  conclusion,  and  recommendations  for  further  research. 

G.  SUMMARY 

This  chapter  provided  a  background  of  the  research  contained  herein,  including 
the  purpose  of  the  research,  the  research  questions,  methodology,  benefits,  and  limitation 
of  the  research.  The  next  chapter  reviews  the  literature  review  associated  with  the 
contract  management  process  and  source  selection  strategies. 
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II.  LITERATURE  REVIEW 


The  purpose  of  this  chapter  is  to  provide  an  overview  to  the  federal  government 
contract  management  process.  This  literature  review  familiarizes  the  reader  with  the  three 
phases  and  six  steps  of  the  contract  management  process,  highlighting  how  each  step 
affects  the  detennination  of  the  source  selection  strategy.  This  chapter  reviews  when  each 
source  selection  strategy  (LPTA  or  Tradeoff)  is  appropriate  according  the  Federal 
Acquisition  Regulation  (FAR)  and  other  guidance.  This  chapter  includes  reports  from 
investigative  agencies  that  show  the  best  value  practices  used  by  various  government 
contracting  agencies  and  the  factors  the  federal  government  considers  when  choosing  a 
source  selection  strategy. 

A.  CONTRACT  MANAGEMENT  STATUTES  AND  REGULATIONS 

To  understand  the  contract  management  process,  one  must  first  know  the  meaning 
of  the  word  contract.  The  FAR  (2014)  defines  a  contract  as 

a  mutually  binding  legal  relationship  obligating  the  seller  to  furnish  the 
supplies  or  services  (including  construction)  and  the  buyer  to  pay  for 
them.  It  includes  all  types  of  commitments  that  obligate  the  Government 
to  an  expenditure  of  appropriated  funds  and  that,  except  as  otherwise 
authorized,  are  in  writing.  In  addition  to  bilateral  instruments,  contracts 
include  (but  are  not  limited  to)  awards  and  notices  of  awards;  job  orders  or 
task  letters  issued  under  basic  ordering  agreements;  letter  contracts; 
orders,  such  as  purchase  orders,  under  which  the  contract  becomes 
effective  by  written  acceptance  or  performance;  and  bilateral  contract 
modifications.  Contracts  do  not  include  grants  and  cooperative  agreements 
covered  by  31  U.S.C.6301,  et  seq.  (FAR,  2014,  Part  2.101b) 

A  slightly  more  understandable  definition  of  a  contract  is  a  written  document  that 
confirms  and  communicates  the  agreement  between  buyer  and  seller,  but  first  and 
foremost  it  helps  develop  and  maintain  professional  business  partnerships  between  the 
two  (Garrett,  2010). 

There  are  several  statutes  and  regulations  that  dictate  how  government  contacts 
are  managed.  The  statutes  include  the  Small  Business  Act  (SBA)  of  1953,  the  Truth  in 
Negotiation  Act  of  1962  (TINA),  the  Competition  in  Contracting  Act  of  1984,  the 
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Federal  Acquisition  Streamlining  Act  of  1994  (FAS A),  and  the  Federal  Acquisition 
Refonn  Act  of  1996  (FARA).  The  primary  regulatory  guidance  for  federal  government 
contracting  is  the  Federal  Acquisition  Regulation  (FAR). 

(1)  The  Small  Business  Act  of  1953 

The  Small  Business  Act  (SBA)  requires  contracting  officers  to  ensure  that  small 
business  concerns  and  small  disadvantaged  business  concerns  obtain  their  fair  share  of 
government  contract  awards  (Nash,  Schooner,  Obrien-DeBakey,  &  Edwards,  2007).  To 
be  considered  a  small  business,  a  company  must  be  “organized  for  profit,”  “have  a  place 
of  business  in  the  United  States,”  “operate  primarily  within  or  make  a  significant 
contribution  to  the  United  States  economy,”  and  be  “independently  owned  and  operated” 
(FAR,  2014,  Part  19.001).  A  small  business  must  not  be  “dominant  in  its  field  on  a 
national  basis”  (SBA,  2015). 

(2)  The  Truth  in  Negotiation  Act  of  1962 

TINA  requires  contracting  officers  to  purchase  supplies  and  services  from 
responsible  sources  at  fair  and  reasonable  prices  (FAR,  2014,  Part  15.403).  TINA 
requires  offerors  to  submit  certified  cost  or  pricing  data  if  a  procurement  exceeds  the 
$700,000  TINA  threshold  (FAR,  2014,  PART  15.403).  Under  TINA,  the  contracting 
officer  obtains  accurate,  complete,  and  current  data  from  offerors  to  establish  a  fair  and 
reasonable  price  (DFAR,  2014,  Part  215.403).  Per  Title  41  U.S.C.  Chapter  35,  TINA 
allows  the  government  to  hold  contractors  financially  and  possibly  criminally  liable  if  it 
is  later  found  that  a  contractor  “did  not  provide  accurate,  complete,  and  current  cost  or 
pricing  data”  (Truth  in  Negotiations  Act,  1962), 

(3)  The  Competition  in  Contracting  Act  of  1984 

CICA  requires  contracting  officers  to  utilize  full  and  open  competition  as  the 
standard.  CICA  provides  certain  exceptions  that  contracting  officers  may  use  instead  of 
full  and  open  competition  (e.g.,  SBA  8(a)  small  or  disadvantaged  businesses,  HUBZone, 
veteran  owned  businesses,  only  one  responsible  source  (FAR,  2014,  Part  6302-1), 
unusual  and  compelling  urgency  (FAR,  2014,  Part  6302-2).  Unless  the  contracting  officer 
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can  justify  an  exception  to  CICA,  full  and  open  competition  must  be  used  in  the 
solicitation  and  awarding  of  federal  government  contracts. 

(4)  The  Federal  Acquisition  Streamlining  Act  of  1994  and  the  Federal 
Acquisition  Refonn  Act  of  1996 

FASA  and  FARA  are  both  designed  to  make  government  contracting  more  like 
the  commercial  sector,  simplifying  the  procurement  process  and  saving  money.  The 
FASA  dictates  a  preference  for  the  use  of  commercial  items  to  fill  the  government’s 
requirements  and  simplifies  the  process  to  acquire  them  through  the  commercial  market. 
When  possible,  acquisitions  should  move  to  a  price-based,  market-driven  environment. 
Source  selection  must  be  made  on  a  “best  value”  not  “cheapest  price”  basis  (Office  of  the 
Under  Secretary  of  Defense  (AT&L),  2011).  The  FARA,  later  called  the  Clinger-Cohen 
Act,  continues  with  FASA’s  preference  for  commercial  procurement  by  eliminating  the 
requirement  for  cost  and  pricing  data  as  well  as  expanding  the  definition  of  commercial 
item  to  include  the  following: 

•  Items  that  have  evolved  from  commercial  items 

•  Items  that  are  commercial  with  modifications  to  meet  government-unique 
requirements 

•  Combinations  of  commercial  items  and  services  for  government  use 

•  Non-developmental  items  (NDI),  or  items  originally  developed  and/or 
sourced  by  a  government  agency 

•  Services  at  catalog  or  market  price 

(E.  Yoder,  personal  communication,  2014) 

These  refonns  expand  the  definition  of  commercial  items  to  encompass  not  only  goods, 
but  also  virtually  all  types  of  services  (Nash  et  ah,  2007). 

(5)  Federal  Acquisition  Regulation 

The  primary  regulation  used  by  all  federal  government  agencies  in  the 
“acquisition  of  supplies  and  services  with  appropriated  funds”  (FAR,  2014)  is  the  FAR.  It 
became  effective  on  April  1,  1984,  and  is  “prepared,  issued,  and  maintained”  within 
applicable  laws  under  the  joint  authorities  of  the  Secretary  of  Defense,  the  Administrator 
of  General  Services,  and  the  Administrator,  National  Aeronautics  and  Space 
Administration  (FAR,  2014,  Part  1.103b).  Government  agencies  also  have  supplements  to 
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the  FAR  with  specific  modifications  that  meet  their  own  requirements.  For  example,  the 
Navy/Marine  Corps  Acquisition  Regulation  Supplement  (NMCARS)  adds  language  to 
the  FAR  specifically  for  Navy  and  Marine  Corps  acquisitions.  The  following  example 
shows  where  the  NMCARS  and  the  FAR  differ. 

(a)(1)  Cost  or  price  evaluation.  Methods  of  evaluation  which  assign  a 
point  score  to  cost  or  price  and  combine  it  with  point  scores  for  other 
evaluation  factors  generally  should  not  be  used  [emphasis  added].  Point 
scores  can  be  helpful  in  summarizing  subjective  evaluation  of  technical 
and  other  factors,  but  are  not  needed  in  evaluating  cost  or  price  and  tend  to 
obscure  the  Tradeoff  between  cost/price  and  other  factors,  rather  than 
clarifying  it.  If  point  scoring  of  cost/price  is  utilized,  it  should  be 
demonstrated  that  the  value  of  a  cost/price  point  is  comparable,  in  value  to 
the  Government,  to  the  value  of  a  non-cost/price  point.  When  a  cost 
realism  analysis  is  performed,  the  resulting  realistic  cost  estimate  should 
be  used  in  the  evaluation  of  cost,  except  when  using  a  firm-fixed-price  or 
fixed-price  with  economic  price  adjustment  type  of  contract.  (NMCARS, 

2014,  5215.305) 

In  the  boldfaced  text,  the  NMCARS  specifically  states  that  the  use  of  a  point  scale 
should  not  be  used  as  an  evaluation  factor  for  cost  or  price,  even  if  used  for  other  aspects 
of  the  Tradeoff  process.  The  rest  of  the  section  refers  to  the  higher  FAR  guidance.  As 
previously  discussed  statutes  and  regulations  dictate  how  government  contacts  are 
managed,  the  next  section  will  discuss  in  more  detail  the  contract  management  process. 

B.  CONTRACT  MANAGEMENT  PROCESS 

The  previous  section  discussed  statutes  and  regulations  governing  the  contract 
management  process.  The  next  section  will  discuss  in  more  detail  the  contract 
management  process  by  looking  at  the  three  phases  and  six  steps.  Contracts  are  looked  at 
from  two  very  different  perspectives.  On  one  side  is  the  seller  (defense  contractors,  in  this 
case),  who  provides  goods  (weapons  systems  or  parts)  or  services  in  return  for  payment. 
The  other  is  the  buyer  (government),  who  purchases  goods  and  services  through  the  use 
of  contracts.  A  definition  for  contract  management  is  “the  art  and  science  of  managing  a 
contractual  agreement  throughout  the  contracting  process”  (Garrett,  2010,  p.  18).  The 
DOD  typically  does  not  produce  any  of  the  items  or  services  it  requires  to  sustain  itself 
through  daily  and  future  operations,  so  it  must  outsource,  or  rely  on  contractors,  to  fulfill 
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its  requirements.  Contracts,  as  stated  previously,  are  legally  binding  documents  that 
require  the  seller  to  provide  the  item  or  service  and  the  buyer  to  pay  for  it,  in  so  reducing 
uncertainty  and  risk  involved  in  transactions.  Contracts  are  the  main  way  to  ensure  both 
the  government  and  contractor  understand  exactly  what  the  agreement  entails,  as  it  is  all 
in  writing.  Understanding  and  adhering  to  the  contract  can  help  develop  and  maintain  a 
professional  and  ethical  relationship  by  both  the  government  and  the  contractor.  Based  on 
Garrett  (2010),  there  are  three  phases  to  the  contract  management  process,  and  within 
those  phases  there  are  six  steps  for  the  buyer  (government)  and  the  seller  (contractor). 
Figure  1  shows  the  flow  of  the  six  steps  through  the  contract  management  process.  For 
the  purpose  of  this  research,  the  focus  is  only  on  the  steps  of  the  buyer. 


Figure  1.  Contract  Management  Process  (from  Garrett,  2010,  p.  20) 


1.  Pre-award  Phase 

In  accordance  with  the  FAR  (FAR,  2014,  Part  7.102)  “all  agencies  will  perform 

acquisition  planning  and  conduct  market  research  on  requirements  determined  to  have 

legitimate  needs.”  The  results  of  that  market  research  will  be  used  to  make  decisions  on 

whether  or  not  the  requirement  will  be  sent  to  prospective  contractors  as  a  request  for 
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proposal/quotation/information  (RFP/RFQ/RFI),  what  factors  will  be  used  to  determine 
best  value  to  the  government,  and  how  the  solicitation  will  be  structured. 

(1)  Step  1:  Procurement  Planning 

The  procurement  planning  step  is  “the  process  of  identifying  which  business 
needs  can  be  best  met  by  procuring  products  or  services  outside  the  organization.  This 
process  involves  determining  whether  to  procure,  what  to  procure,  how  to  procure,  and 
when  to  procure”  (Garrett,  2010,  81).  As  this  is  the  initial  step  and  stage,  it  is  where  you 
are  planning  the  various  aspects  of  the  procurement  (e.g.,  source  selection  strategy, 
contract  type,  contract  structure,  etc.)  that  will  be  used  throughout  the  entire  contract 
management  process.  The  initial  analysis  of  the  requirement,  or  whether  to  “build  or 
buy,”  is  done  during  this  stage  to  determine  what  exactly  is  needed,  be  it  a  service  or 
weapon  system.  Once  the  detennination  is  concluded,  market  research  must  be  conducted 
to  find  the  most  suitable  approach  to  acquiring  the  requirement.  Market  research  may  be 
done  though  pre-solicitation  conferences  or  other  means.  Risk  analysis  is  conducted  to 
detennine  the  ability  of  industry  to  handle  the  requirement,  the  cost  effectiveness  of 
going  forward  with  the  proposal,  and  the  best  source  selection  strategy  to  use.  A 
procurement  management  plan  is  created  that  describes  the  procurement  process 
throughout  the  management  of  the  contract.  The  development  of  an  Invitation  for  Bid 
(IFB)  or  Request  for  Proposal  (RFP)  will  then  be  drafted,  as  will  preliminary  documents 
to  be  used  in  the  solicitation  planning  step. 

(2)  Step  2:  Solicitation  Planning 

The  solicitation  planning  step  is  “the  process  of  preparing  the  documents  needed 
to  support  the  solicitation”  (Garrett,  2010,  p.  88).  Prior  to  beginning  this  step,  the 
procurement  management  plan  will  be  reviewed.  The  statement  of  work  (SOW)  will  be 
developed  to  explain  in  clear,  concise  language  the  work  the  contractor  must 
accomplish. 1  The  SOW  may  be  considered  the  most  critical  document  in  the  acquisition 
process  (Rumbaugh,  2010).  Individuals  from  both  the  government  and  industry  read  the 

1  Statement  of  objectives  (SOO),  work  breakdown  structures  (WBS),  and  performance  work  statement 
(PWS)  may  be  used  as  well  or  in  place  of  SOW  depending  on  contract  type 
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SOW;  therefore,  if  it  does  not  reflect  exactly  the  requirements  as  intended,  severe  issues 
in  the  contract  administration  process  may  occur.  The  SOW  sets  the  bounds  of  the 
government’s  requirements,  enabling  the  contractor  to  accomplish  them.  The  SOW 
reflects  the  results  of  market  research,  and  impacts  all  other  steps  of  the  contracting 
process.  During  this  step,  proposal  evaluation  factors  are  developed  to  communicate 
program  priorities  in  general  order  of  importance.  Some  examples  of  proposal  evaluation 
factors  include:  technical  perfonnance,  past  perfonnance,  cost,  supportability, 
producibility,  and  management  approach.  The  requirements  of  the  SOW  will  be  related  to 
the  source  selection  strategy,  whether  it  is  LPTA  or  Tradeoff.  The  solicitation  planning 
step  output  is  the  solicitation,  which  is  in  the  form  of  an  RFP  or  IFB.  If  the  source 
selection  strategy  selected  is  anything  other  than  LPTA,  then  the  solicitation  must  be  in 
the  form  of  an  RFP,  as  IFBs  do  not  allow  for  anything  other  than  LPTA  and  also  do  not 
allow  for  any  type  of  communication  with  the  bidders  during  the  source  selection  process 
(FAR,  2014,  Part  14.101).  The  solicitation  will  include  the  SOW  and  any  other  clarifying 
documentation  such  as  instructions  to  offerors  regarding  how  to  complete  and  submit 
their  proposal.  When  considering  source  selection  strategy,  Section  M  of  the  solicitation 
is  the  key  section  that  dictates  how  the  proposals  will  be  evaluated  and  what  the 
evaluation  criteria  will  be,  whether  an  LPTA  or  Tradeoff  approach  will  be  used  (FAR, 
2014,  Part  15.304e).  Section  M  identifies  all  significant  evaluation  factors  that  will  be 
considered  in  evaluating  proposals  and  their  relative  importance.  When  evaluation  factors 
other  than  cost/price  are  used,  the  solicitation  must  state  if  the  evaluation  factors  are 
significantly  more,  approximately  equal,  or  significantly  less  important  than  cost/price.  A 
solicitation  can  have  Tradeoff  factors  that  are  more  advantageous  to  the  federal 
government  and  still  award  to  the  offeror  with  the  lowest  bid  submitted.  With  all  of  the 
infonnation  derived  from  solicitation  planning  step,  procurement  documents  can  be 
developed. 

(3)  Step  3:  Solicitation 

The  solicitation  process  “consists  of  obtaining  infonnation  (bids  and  proposals) 
from  prospective  sellers  on  how  project  needs  can  be  met”  (Ganett,  2010,  p.  90).  In  the 
previous  step,  bidders  would  have  been  identified,  and  a  qualified  bidders  list  would  be 
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generated.  It  is  important  to  ensure  that  all  bidders  have  a  clear  understanding  of  what  the 
government  is  requesting  in  the  RFP.  In  many  solicitations,  the  government  will  advertise 
through  the  government-wide  point  of  entry  (GPE),  “the  single  point  where  government 
business  opportunities”  which  exceed  $25,000,  including  synopses  of  proposed  contract 
actions,  solicitations,  and  associated  information,  can  be  accessed  electronically  by  the 
public”  (FAR,  2014,  Part  2.101).  The  GPE  is  located  at  www.fedbizops.gov.  The 
resulting  fonnal  bids  should  be  submitted  by  responsible  sources  (FAR,  2014,  Part 
9.103). 


2.  Award  Phase 

During  the  award  phase,  the  government  chooses  which  perspective  contractor 
will  receive  the  contract  award.  Both  the  government  and  contractor  may  participate  in 
contract  negotiations,  in  which  tenns  and  conditions  are  agreed  upon  to  ensure  both 
parties  achieve  their  goals,  to  include  cost,  schedule,  performance,  and  any  other 
requirement  of  the  contract. 

(4)  Step  4:  Source  Selection 

The  source  selection  process  “entails  receiving  bids  or  proposals  and  applying 
evaluation  criteria  to  select  a  provider”  (Garrett,  2010,  p.  137).  There  are  different 
approaches  to  handle  the  source  selection  process  based  on  the  type  of  procurement.  If  it 
is  sealed  bid  procurement,  then  the  bid  evaluation  is  a  very  structured  process.  As  long  as 
the  bid  conforms  to  what  is  in  Section  M,  then  price  is  the  only  factor  considered.  For 
negotiated  contracts,  evaluation  criteria  would  have  been  established  in  the  solicitation 
planning  step.  The  criterion  needs  to  ensure  the  greatest  potential  for  successful 
performance  or  cost  effectiveness  for  the  government.  Best  value  depends  upon  “sound 
selection  strategies  that  ensure  that  the  outlined  project  procurement  objectives,  including 
client/user  demands  are  met”  (Palaneeswaran,  Kumaraswamy,  &  Ng,  2003).  The  best 
value  continuum  states,  “An  agency  can  obtain  best  value  in  negotiated  acquisitions  by 
using  any  one  or  a  combination  of  source  selection  approaches.  In  different  types  of 
acquisitions,  the  relative  importance  of  cost  or  price  may  vary”  (FAR,  2014,  Part  15.101). 
An  example  of  an  acquisition  where  price  may  not  be  as  important  is  an  item  that  is  well 
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defined  and  has  relatively  low  performance  risk,  like  furniture  or  tires.  These  items  could 
have  cost/price  as  the  deciding  factor.  An  item  like  a  new  radar  system  for  an  AEGIS - 
class  destroyer  requires  much  more  technical  development  and  is  far  harder  to  adequately 
define.  In  this  type  of  acquisition,  factors  like  past  perfonnance  and  technical 
performance  might  be  more  important  than  cost.  Clearly,  the  choice  of  using  either  the 
LPTA  or  Tradeoff  strategy  plays  a  role  in  the  government’s  ability  to  choose  the  offeror 
that  presents  the  “best  value.”  Figure  2  illustrates  the  relative  importance  of  cost/price 
factors  and  non-cost/non-price  factors  for  each  source  selection  strategy. 


Non-cost  factors 
less  important 
than  price 

Non-cost  factors 
equal  to  price 

Non-cost  factors 
more  important 
than  price 

Lowest  price  technically 
acceptable  process 


Tradeoff  process 


y 


When  proposals  are  deemed 
technically  acceptable,  price  is 
the  determining  factor 


Requirements  clearly  defined 
Development  work  low 
Contract  performance  risk  low 


Importance  of  price  decreasing 


Requirements  less  defined 
►  Development  work  high 
Contract  performance  risk  high 


Source  GAO  analysis  ol  Federal  Acquisition  Regulation  Part  15  101  |  GAO-14-564 


Figure  2.  Best  Value  Continuum  (from  GAO,  2010) 


In  LPTA,  an  established  acceptability  standard  is  set  for  technical  requirements, 
and  that  is  what  all  bidders  must  meet.  The  contract  is  awarded  to  the  bidder  who  meets 
that  standard  (a  standard  considered  technically  acceptable)  and  has  the  lowest  price. 
Tradeoff  selections  occur  when  the  contracting  officer  believes  that  the  offer  gives  the 
government  an  advantage  somewhere  else  on  the  best  value  continuum  that  is  not  strictly 
based  on  cost/price,  though  cost  is  always  a  factor. 

3.  Post-award  Phase 

The  post-award  phase  includes  events  that  take  place  once  the  contract  has  been 
awarded  through  its  completion,  whether  that  is  by  completion  or  tennination.  No  two 
contracts  are  alike  in  how  they  are  administered  once  a  contract  is  awarded;  contract 
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administration  can  be  direct  and  simple  or  quite  complex.  The  key  is  in  monitoring  and 
evaluating  the  contractor  to  ensure  that  the  terms  and  conditions  of  the  contract  are  being 
met.  Contract  closeout  entails  taking  care  of  the  final  details  of  the  contract  so  it  can  be 
closed  out. 

(5)  Step  5:  Contract  Administration 

The  contract  administration  step  begins  once  the  contract  is  awarded;  it  is  “the 
process  of  ensuring  that  each  party’s  perfonnance  meets  contractual  requirements” 
(Garrett,  2010,  p.  162).  Much  of  the  documentation  and  information  developed  and 
determined  early  in  the  solicitation  planning  step,  such  as  the  SOW,  terms  and  conditions, 
and  schedule  are  executed  during  this  step.  The  contract  itself  is  a  crucial  piece  to  guide 
the  effort.  Oftentimes  changes  or  modifications  are  necessary  when  dealing  with 
government  contracts;  these  take  place  during  the  contract  administration  step.  There  are 
two  types  of  modifications  listed  in  the  FAR.  A  bilateral  change  is  something  that 
requires  the  agreement  of  both  parties  and  typically  includes  a  change  that  requires  some 
sort  of  equitable  adjustment  (i.e.,  agreed-upon  changes  to  contract  price,  schedule,  etc.). 
Unilateral  changes  only  require  the  contracting  officer’s  approval  and  are  typically  used 
for  administrative  issues  (e.g.,  changes  in  accounting  lines,  payment  processes,  etc.).  The 
contract  administration  step  is  where  the  contractor  is  actually  doing  what  the 
government  is  paying  for,  so  it  is  imperative  that  the  contractor’s  work  is  monitored  to 
ensure  that  what  was  detennined  in  the  source  selection  strategy,  whether  it  was  an 
established  acceptable  standard  or  the  highest  technical  performance,  is  being  met.  There 
are  several  means  to  monitor  the  contractor’s  perfonnance.  The  government  may  use 
quality  assurance  evaluators  (QAE)  or  contracting  officer’s  technical  representatives 
(COTR)  to  ensure  the  contractor  is  providing  materials  or  services  at  an  acceptable 
standard.  These  individuals  provide  the  contracting  officer  with  information  to  ensure  the 
contractor  is  meeting  the  requirements  of  the  SOW  and  other  contractual  requirements. 
Another  way  to  measure  the  contractor’s  performance  is  thru  earned  value  management 
(EVM).  EVM  measures  a  project’s  cost,  schedule  and  performance  progress  by 
comparing  the  actual  figures  with  planned  estimates.  Perfonnance  is  measured  by  taking 
the  budgeted  cost  of  work  performed  and  comparing  it  to  the  actual  costs  of  the  work 
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performed  (Rendon  &  Snider,  2008).  Withholding  contractor  payments  can  also  be  used 
if  perfonnance  is  lacking,  but  this  should  not  be  done  unless  all  other  means  have  been 
exhausted  and  not  without  following  all  applicable  guidelines. 

(6)  Step  6:  Contract  Closeout  and  Tennination 

Contract  closeout  and  termination  is  the  final  step  for  any  contract  and  can  go  in 
three  different  ways:  successful  perfonnance,  tennination  for  convenience  (T4C),  or 
tennination  for  default  (T4D).  A  successful  contract  closeout  is  how  the  majority  of 
contracts  conclude.  This  occurs  when  the  contractor  has  completed  all  assigned  tasks  and 
they  are  accepted  by  the  buyer  (Ganett,  2010,  p.  185).  This  means  that  all  items  or 
services  on  the  contract  have  been  received  and/or  met  and  the  contractor  has  received 
final  payment.  Upon  receipt  of  the  contract  completion  statement,  the  contracting  officer 
initiates  the  contract  closeout  process  in  accordance  with  FAR  Part  4.804  (Rendon  & 
Snider,  2008,  p.  180). 

Contracts  that  are  not  completed  are  closed  out  through  termination,  either  for 

convenience  or  for  default.  T4C  happens  when  the  government  exercises  its  unique  right 

to  terminate  either  partially  or  completely  any  contract  unilaterally.  This  process  must  be 

in  writing,  and  there  is  no  requirement  for  justification.  The  contractor  is  entitled  to 

tennination  costs  associated  with  the  contract,  which  may  include  compensation  for  work 

done  and  allowance  for  profit  (FAR,  2014,  Part  49.2).  T4D  occurs  when  the  government 

exercises  its  right  to  tenninate  a  contact  either  “partially  or  completely  due  to  the 

contractor’s  actual  or  anticipated  failure  to  perform  contractual  obligations”  (FAR,  2014, 

Part  49.401).  Any  contractor  who  receives  a  T4D  may  be  liable  to  the  government  for 

any  funds  received,  regardless  of  work  performed,  and  any  excess  costs  due  to 

procurement.  The  contractor  may  also  be  liable  for  procurement  costs,  should  the 

government  need  to  award  a  contract  to  a  different  contractor  to  complete  the  required 

work.  A  T4D  can  also  affect  the  contractors  past  performance  evaluation  for  future 

contracts.  It  is  during  the  contract  closeout  step  in  which  the  government  conducts  an 

assessment  of  the  contractors  performance  using  the  Contract  Perfonnance  Assessment 

Reporting  System  (CPARS)  and  Past  Perfonnance  Information  Retrieval  System  (PPIRS) 

to  ensure  past  perfonnance  information  is  submitted  properly  and  within  an  appropriate 
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time  (FAR,  2014,  Part  42.1501).  As  previously  discussed,  the  source  selection  step  is 
where  bids  or  proposals  are  received  based  on  evaluation  criteria,  the  next  section  will 
discuss  in  more  detail  the  two  types  of  source  selection  strategies. 

C.  SOURCE  SELECTION  STRATEGY 

Source  selection  strategies  consist  of  LPTA  as  well  as  Tradeoff.  The  following 
discussion  provides  additional  detail  for  each  of  these  strategies. 

1.  Lowest  Price  Technically  Acceptable 

Contracting  officers  use  the  LPTA  source  selection  strategy  when  “best  value  is 
expected  to  result  from  selection  of  the  technically  acceptable  proposal  with  the  lowest 
evaluated  price”  (FAR,  2014,  Part  15.101-2).  When  using  LPTA  as  a  source  selection 
strategy,  the  solicitation  detennines  evaluation  factors  and  significant  subfactors  that 
establish  the  requirements  of  acceptability.  Solicitations  specify  “the  award  will  be  made 
on  the  basis  of  the  lowest  evaluated  price  of  proposals  meeting  or  exceeding  the 
acceptability  standards  for  non-cost  factors”  (FAR,  2014,  Part  15.101-2).  Contracting 
officerss  are  not  allowed  to  use  of  tradeoffs  in  LPTA  actions.  Non-cost/price  factors  are 
not  used  during  evaluation  for  acceptability  (FAR,  2014,  Part  15.101-2).  This  means  that 
once  the  government  determines  an  acceptable  minimum  standard,  it  can  still  look  at 
other  factors,  but  cannot  use  any  of  them  as  a  deciding  factor  for  making  its  decision  as  to 
who  will  receive  the  contract.  Once  technical  acceptability  is  established,  the  only 
relevant  factor  is  cost/price.  LPTA  is  typically  used  for  contracts  that  have  low 
performance  risk  and  well-defined  requirements  (FAR,  2014,  Part  15.101). 

2.  Tradeoff 

Contracting  officers  use  the  Tradeoff  source  selection  strategy  when  it  is  expected 
that  the  best  value  can  be  received  from  other  than  the  lowest  priced  offeror  or  other  than 
the  highest  technically  rated  offeror.  According  to  the  FAR  any  items  that  may  affect  the 
contract  award  for  a  Tradeoff,  evaluation  factors  and  significant  subfactors,  must  be 
clearly  stated  in  the  solicitation  with  their  relative  importance.  The  solicitation  states 
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whether  all  evaluation  factors  other  than  cost  or  price,  “when  combined,  are  significantly 
more  important  than,  approximately  equal  to,  or  significantly  less  important  than  cost  or 
price”  (FAR,  2014,  Part  15.101).  Through  this  process,  the  government  may  choose  a 
proposal  that  is  other  than  the  lowest  price.  “The  perceived  benefits  of  the  higher  priced 
proposal  shall  merit  the  additional  cost,  and  the  rationale  for  Tradeoffs  must  be 
documented”  (FAR,  2014,  Part  15.101-1).  This  means  that  if  the  source  selection  team 
detennines  that  factors  other  than  price  will  provide  a  better  value  to  the  government, 
then  the  contracting  officer  may  use  those  as  the  primary  factors  when  awarding  the 
contract.  Contracting  officers  must  always  remember  that  they  have  a  responsibility  to 
the  tax  payers  and  ensure  that  the  factors  provide  the  best  value.  In  a  situation  where 
technical  perfonnance  is  weighted  higher  than  cost/price,  an  offeror  that  is  the  highest 
technically  rated  may  still  lose  the  contract  to  a  lower  priced  bid  if  that  offeror  is  deemed 
to  be  economically  out  of  range  relative  to  other  closely  matched  technical  offerors.  The 
government,  even  in  a  Tradeoff  strategy,  may  award  to  the  lowest  price  offeror  that  meets 
the  technical  requirements.  Even  though  the  source  selection  strategy  is  based  on  a 
Tradeoff,  the  contracting  officer  must  still  explain  why  the  winning  bid  won  (i.e.,  provide 
a  rationale  for  award)  if  it  was  higher  priced  than  other  bids.  This  is  easily  done  if  the 
winning  contractor  is  far  superior  technically  and  close  in  price,  but  if  the  technical 
performance  is  close  and  the  price  has  a  large  disparity,  this  may  cause  issues,  including  a 
protest  from  a  lower  bidder.  A  few  key  factors  to  ensure  a  successful  use  of  a  Tradeoff 
source  selection  are  training  team  members  involved  and  a  clear  source  selection  plan.  It 
is  imperative  to  have  a  motivated  and  technically  talented  team  led  by  a  strong  leader  that 
understands  the  objectives  of  the  contract.  The  team  needs  a  well-published  schedule, 
and  must  have  open  dialogue  and  respect  between  all  team  members  (Wydler,  2010).  The 
previous  discussion  covered  the  specifics  of  the  source  selection  process  and  the  two 
source  selection  strategies,  LPTA  and  Tradeoff;  the  next  section  will  identify  some 
agency  reports  on  how  these  source  selection  policies  have  been  implemented. 
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D.  INVESTIGATIVE  REPORTS 


Since  1992,  GAO  has  classified  DOD’s  contract  management  as  a  high  risk.  The 
challenges  DOD  faces  according  to  GAO  are:  an  insufficient  acquisition  workforce, 
ineffective  contracting  techniques  and  approaches,  less  established  procedures  for  service 
acquisitions,  and  operational  contract  support.  Of  the  areas  of  concern  highlighted  by  the 
report,  the  area  most  directly  related  to  the  source  selection  process  is  the  contracting 
techniques  and  approaches.  According  to  GAO,  DOD  has  struggled  with  utilizing  the 
most  effective  contract  type  and  the  effective  use  of  competition  (GAO,  2015). 
Consequently,  Congress  has  shown  an  increased  focus  on  the  use  of  best  value  processes 
in  the  DOD  acquisition  process.  Through  Section  845  of  the  National  Defense 
Authorization  Act  (NDAA)  for  fiscal  year  2010,  Congress  mandated  the  GAO  to  review 
the  “DOD’s  use  of  the  best  value  Tradeoff  process,  specifically  when  non-cost  factors 
were  more  important  than  price”  (National  Defense  Authorization  Act,  2010).  The  GAO 
looked  at  fiscal  year  2009  contracts  with  a  minimum  dollar  value  of  $25  million  due  to 
the  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS)  requirement  of 
contracts  with  a  value  of  $25  million  or  more  in  a  fiscal  year  to  have  a  written  acquisition 
plan  (DFARS  207.103).  The  GAO’s  research  looked  at  (a)  how  often  and  for  what  types 
of  contracts  the  DOD  used  the  best  value  Tradeoff  process;  (b)  why  and  how  the  DOD 
used  such  an  approach;  and  (c)  challenges,  if  any,  the  DOD  faced  in  using  the  best  value 
Tradeoff  process  (GAO,  2010).  The  GAO  found  that  the  DOD  used  best  value  processes 
(LPTA  or  Tradeoff)  in  95%  of  the  competitively  awarded  contracts  it  reviewed.  Figure  3 
shows  the  different  source  selection  strategies  the  GAO  found  in  its  research  and  the 
percentage  breakdown  between  cost  and  non-cost  factors.  In  69%  of  awarded  contracts, 
Tradeoff  was  the  source  selection  strategy  used. 
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Source:  GAO  analysis  ol  DOD  contract  documents 

Figure  3.  Selection  Strategy  Percentages  (from  GAO,  2010,  p.  7) 

When  the  DOD  ranked  Tradeoff  (non-cost  factors  more  important  than  price),  the 
typical  factors  were  (not  necessarily  in  this  order)  past  performance,  technical,  small 
business,  experience,  and  management.  The  GAO  found  that  in  Tradeoff  source 
selections,  the  DOD  selected  a  lower  priced  proposal  almost  as  often  as  it  selected  the 
non-lowest  cost  proposal.  In  the  situations  in  which  the  DOD  chose  an  offer  without  the 
lowest  cost,  on  average,  the  cost  differential  was  less  than  5  percent.  There  were 
situations  where  the  cost  differential  was  much  higher,  as  in  the  Marine  Corps  paying 
48%  more  than  the  next  lower  bidder  for  burn-resistant  clothing  for  soldiers  in  Iraq.  In 
that  situation,  the  benefit  of  greater  second-  and  third-degree  burn  protection  outweighed 
the  next  offeror’s  proposal  (GAO,  2010,  p.  15).  The  DOD  did  state  that  when  using 
Tradeoff  as  a  source  selection  strategy,  there  were  certain  challenges  that  may  arise  that 
were  not  as  prevalent  in  the  LPT  A  strategy,  such  as  difficulties  in  developing  meaningful 

evaluation  factors,  additional  time  investment,  and  the  level  of  business  judgment 
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required.  Instances  arose  where  contracting  personnel  awarded  a  contract  based  on 
Tradeoff  procedures,  but  failed  to  adequately  evaluate  the  factors.  The  Naval  Facilities 
Engineering  Command  (NAVFAC)  was  investigated  by  the  Department  of  Defense 
Inspector  General  (DODIG)  over  a  contract  that  in  the  end  revealed  the  correct 
prospective  contractor  won  the  bid,  but  there  were  issues  in  the  technical  evaluation  and 
missed  past  perfonnance  factors  (DODIG,  2006).  The  Marine  Corps  experienced  a 
similar  situation  where  evaluation  criteria  in  solicitations  lacked  adequate  documentation 
and  the  disclosure  of  technical  data.  (DODIG,  2009).  These  difficulties  are  exasperated 
by  the  knowledge  that  many  seasoned  and  experienced  acquisition  professionals  are 
retiring  and  the  DOD  expects  to  increase  its  contracting  career  field  by  6,400  personnel 
by  fiscal  year  2015  (GAO,  2010,  p.  18). 

The  GAO  concluded  that  in  fiscal  year  2009,  best  value  Tradeoff  accounted  for 
the  majority  of  competitive  contract  awards,  and  using  the  process  effectively  depends  on 
making  sound  Tradeoffs  between  price  and  non-cost  factors.  It  recommended  the  DOD 
develop  a  training  plan  to  help  contracting  professionals  detennine  when  the  price 
differential  is  warranted  in  making  Tradeoff  award  decisions.  The  DOD  concurred  with 
this  assessment  (GAO,  2010,  p.  26). 

The  National  Defense  Authorization  Act  of  2014  again  mandated  that  the  GAO 
review  the  DOD’s  use  of  best  value  contract  award  processes.  This  time  the  GAO  looked 
at  contracts  with  obligations  over  $1  million,  the  DOD  and  military  departments’ 
guidance  on  use  of  best  value,  and  training  provided  to  acquisition  professionals  from  the 
Defense  Acquisition  University  (DAU)  and  military  departments.  The  GAO  found  that, 
again,  Tradeoff  was  used  in  the  majority  of  contracts.  LPTA  was  used  most  often  for 
higher  dollar  obligations  to  acquire  commercial  products  and  for  lower  dollar  obligations 
to  acquire  both  products  and  services.  From  2009  to  2013,  the  use  of  LPTA  increased  by 
9%  in  contracts  over  $25  million  (GAO,  2014,  p.  10).  Figure  4  shows  the  changes 
between  the  two  GAO  reports.  The  DOD’s  increased  ability  to  appropriately  define  its 
requirements  and  its  knowledge  of  potential  contractors,  through  market  research,  were 
key  factors  in  detennining  when  to  use  Tradeoff  or  LPTA. 
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Table  1.  GAO  Analysis  of  DOD  Solicitation  Documents 

(from  GAO,  2014,  p.  12) 


Source  selection 
process 

Fiscal  year  2009 
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Fiscal  year  2013 
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69 
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Lowest  price  technically 
acceptable 

26 

36 
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Sealed  bid 

5 

6 

No 

Source  2009  GAO  1 1  S:  2013  GAO  analysis  of  DOD  soucrtalton  documents  |GAO-1A-5W 


Declining  budgets  also  increased  the  use  of  LPTA  source  selection  strategy 
(GAO,  2014,  p.  12).  The  GAO  found  that  the  DAU  and  military  departments  provide 
classroom  and  online  training  related  to  source  selection,  though  it  is  stressed  by  both  that 
on-the-job  training  is  key  for  acquisition  personnel  to  make  infonned  source  selection 
decisions.  Defense  Procurement  and  Acquisition  Policy  officials  are  also  working  on 
updating  the  DOD’s  source  selection  procedures  to  further  define  how  to  conduct  best 
value  source  selections  (GAO,  2014,  p.  16). 

Though  the  two  GAO  reports  showed  that  Tradeoff  is  chosen  significantly  more 
than  LPTA,  there  are  many  in  the  defense  industry  who  see  the  LPTA  strategy  as  the 
DOD  attempting  to  meet  their  requirements  in  the  least  expensive  manner,  and,  as  a 
result,  the  warfighter  suffers.  Ed  Spitler,  president  of  Astrium  Services  Government  Inc., 
a  part  of  the  Airbus  Group,  said  “the  U.S.  military’s  ‘lowest  price,  technically  acceptable’ 
procurement  strategy  is  stifling  innovation  and  ultimately  shortchanging  war  fighters” 
(Magnuson,  2014).  He  also  said,  “You  may  have  written  the  best  proposal  of  your  life  ... 
but  it  will  never  be  read  by  the  government  because  of  that  LPTA  requirement.  It’s  a 
shame  because  there  is  no  room  for  innovation  when  you  do  that.”  Mr.  Spitler  says  he  has 
asked  procurement  officials  how  they  can  give  contracts  to  such  small,  inexperienced 
firms,  and  their  attitude  is:  “If  they  fail  or  default  in  year  one,  we  just  recompete  it” 
(Magnuson,  2014).  In  another  article,  Bob  Lohfeld  of  the  Lohfeld  Consulting  Group 
explained  how  in  an  LPTA  contract  award;  factors  other  than  price  cannot  be  used  in  the 
evaluation  if  the  product  is  deemed  technically  acceptable.  This  means  that  when 
applying  LPTA  criteria  to  past  performance  evaluations,  an  “offeror  without  a  record  of 
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relevant  past  performance  or  for  whom  information  on  past  perfonnance  is  not  available 
or  is  so  sparse  that  no  meaningful  past  perfonnance  rating  can  be  reasonably  assigned; 
the  offeror  may  not  be  evaluated  favorably  or  unfavorably  on  past  performance”  (FAR, 
2014,  Part  15.305).  Therefore,  the  offeror  shall  be  detennined  to  have  unknown  past 
perfonnance.  In  the  context  of  acceptability/unacceptability,  unknown  shall  be 
considered  acceptable  (Lohfeld,  2012). 

John  Coombs,  fonner  Assistant  Secretary  of  the  Anny  for  Acquisition,  Logistics 
and  Technology,  wrote  that  the  way  to  get  LPTA  right  is  by  raising  the  bar  for  technical 
acceptability.  He  wrote  that  in  an  LPTA  awarded  contract,  prospective  contractors 
receive  nothing  for  having  more  qualified  personnel.  If  all  have  met  the  technically 
acceptable  standard,  those  personnel  will  cost  more  and  mean  someone  else  may  have  the 
lowest  price.  By  setting  the  technically  acceptable  skill  level  in  the  requirement,  that 
becomes  the  minimum  skill  required  for  the  contractor  (Coombs,  2013).  This  can  help 
achieve  a  technical  acceptance  that  is  based  more  on  the  non-cost/non-price  factors  than 
just  the  price  of  the  bids  received. 

Jacques  S.  Gansler,  former  undersecretary  of  defense  for  acquisition,  technology, 
and  logistics,  and  William  Lucyshyn,  researched  the  DOD’s  use  of  LPTA  as  a  source 
selection  strategy.  They  found  it  was  imperative  for  government  acquisition  professionals 
to  leverage  LPTA  effectively  and  appropriately  to  find  cost  savings  with  anticipated 
declining  budgets.  They  warned  that  LPTA  overuse  could  have  drawbacks;  lack  of 
innovation  by  contractors,  impact  on  quality,  and  lowered  investment  on  human  capital. 
Gansler  and  Lucyshyn  also  provided  some  recommendations  on  ways  to  improve  the  use 
of  LPTA  source  selection,  without  stifling  innovation  and  risking  project  completion. 
Only  use  LPTA  when  “technically  acceptable”  can  be  fully  defined  and  the  risk  is  low. 
Past  perfonnance  should  be  used  as  an  evaluation  factor  in  the  LPTA  process,  especially 
when  acquiring  complex  mission  support  services.  LPTA  should  be  used  when  items  can 
be  clearly  defined,  but  the  risk  of  over  using  LPTA  to  save  money  could  have  effects  on 
what  the  warfighter  receives  in  quality,  reduced  industry  innovation,  and  reduced 
schedule  and  delivery  accuracy.  The  quality  of  government  and  prospective  contractor 
communications  needs  to  be  improved.  Unequal  communications,  or  the  lack  of 
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communications,  can  lead  to  bid  protests  and  misunderstandings  of  the  government’s 
showneeds.  The  last  recommendation  the  two  make  is  for  the  government  to  invest  in  the 
acquisition  work  force.  In  situations  where  LPTA  use  was  criticized,  it  is  sometimes 
unclear  whether  the  LPTA  source  selection  strategy  itself  was  to  blame,  or  if  the  problem 
rests  with  the  government’s  inability  to  sufficiently  identify  and  articulate  the  minimum 
requirements.  Source  selection  requires  a  highly  trained  workforce  to  detennine  if 
LPTA  is  the  appropriate  strategy  (Gansler  &  Lucyshyn,  2013). 

E.  SUMMARY 

This  chapter  presented  an  overview  of  the  contract  management  process  and  the 
associated  literature.  The  chapter  began  with  the  basics  of  contract  management  and  also 
showed  statues  and  regulations  associated  with  it.  We  then  discussed  the  three  phases  of 
contract  management  as  well  as  the  six  steps  of  the  contract  management  process, 
highlighting  how  each  had  an  effect  on  the  source  selection  strategy.  We  then  focused  on 
the  two  source  selection  strategies  and  their  differences.  The  next  chapter  provides  a 
discussion  of  the  major  Navy  acquisition  commands,  NAVAIR  and  NAVSEA,  and  why 
they  were  chosen  for  this  research. 
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III.  NAVAL  SYSTEMS  COMMANDS 


This  chapter  provides  an  overview  of  the  DOD  and  Navy  acquisition 
organizations  and  the  two  major  Naval  Systems  Commands:  NAVAIR  and  NAVSEA. 
The  research  provides  information  on  the  commands’  organization,  missions,  and 
contracting  departments. 

A.  DOD  ACQUISITION  ORGANIZATION 

The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  (USD 
[AT&L])  is  the  principal  staff  assistant  (PSA)  to  the  Secretary  and  Deputy  of  Defense  for 
all  matters  relating  to  the  DOD  acquisition  system  (USD  [AT&L],  2014).  A  primary 
function  of  USD  (AT&L)  is  to  produce  the  DOD  directive  5000.01.  DOD  directive 
5000.01  is  the  principle  directive  on  defense  acquisition  and  provides  policy  and 
guidance  for  all  DOD  acquisition  programs.  The  primary  mission  of  Defense  Acquisition 
is  to  produce  quality  and  cost  efficient  products  that  satisfy  user  needs  to  accomplish  the 
mission.  There  are  five  major  policy  objectives  of  Defense  acquisition:  promotion  of 
competition;  realistic  cost  projections;  affordability,  the  reality  of  fiscal  constrains; 
knowledge-based  acquisition  to  include  the  reduction  of  manufacturing  risk  and 
demonstration  of  producibility;  and  application  of  a  systems  engineering  process 
(Acquisition  Community  Connection,  2014).  Figure  4  demonstrates  that  the  DOD 
acquisition  organization  is  a  multi-tier  management  structure.  Below  the  USD  (AT&L) 
are  the  service  acquisition  executives  (SAE),  who  oversee  the  program  executive  offices 
(PEO).  PEO  consists  of  the  program  management  offices,  which  are  headed  by  program 
managers  (PM)  and  include  other  members  of  the  acquisition  team.  The  Navy’s 
Acquisition  Organization  will  be  discussed  next. 


25 


Not  Sbou.  *r«j  Spu*  A  S'uitiic  Oeleise  Css ■> id.  US  Uifiie  Corps  tid  Otltut 


Assistance 

Secretary 

/nm  \  ' 


•Array 

Material 

Connand 


—  PROGRAM  REPORTING 
"Tl  COO  RO /SUPPORT 

—  COMMAND 


PROGRAM  MANAGER'S  REPORTING  CHAIN 
(WEAPON  SYSTEM/EXAMPIE) 


USD  (A&T) 


ARMY 

ACQUISITION 

EXECUTIVE 


Assistance 

Secretary 

(ALT)' 


NAVY 

ACQUISITION 

EXECUTIVE 


OEfENSE  ACQUISITION 
EXECUTIVE 


AIRFORCE 

ACQUISITION 

EXECUTIVE 


Assistance 

Secretary 


CSA 

r"  l 

i 

i 

CNO 

r"  i 

i 

i 

CSAf 

Muiffrs 


S'. 


irrad 


{IS  tics 


Figure  4.  USD  (AT&L)  Acquisition  Community  Connection  (from 
Acquisition  Community  Connection,  2014) 


B.  NAVY  ACQUISITION  ORGANIZATION 

The  Assistant  Secretary  of  the  Navy  (ASN)  for  research,  development  and 
acquisition  (RDA)  manages  the  Navy  acquisition  organization.  ASN  (RDA)  is 
responsible  for:  representation  before  AT&L  and  Congress  regarding  policies  and 
programs;  development  of  Marine  Corps  and  Navy  weapons  systems;  and  acquisition  for 
entire  functions  and  programs.  Different  layers  within  the  Navy’s  organization  taskforce 
execute  its  mission:  ASN;  PEOs;  direct  reporting  program  managers  (DRPMs);  the  naval 
systems  commands;  and  their  field  activities.  Each  layer  carries  out  its  own 
responsibilities.  For  example,  PEOs  and  DRPMs  overlook  different  systems  commands 
(SYSCOMs)  such  as  Naval  Sea  Systems,  Naval  Air  Systems,  Space  and  Naval  Warfare, 
and  Naval  Supply  Systems.  The  SYSCOMs  primarily  support  NAVY  with  material  and 
they  report  to  ASN  (Acquisition  Community  Connection,  2014). 

Figure  5  shows  the  Navy  organizational  chart  for  research,  development,  and 
acquisition. 
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Figure  5.  Navy  Organization  Chart  for  Research,  Development,  and  Acquisition  (from  ASN  [RDA],  2014) 


27 


C.  DEPUTY  ASSISTANT  SECRETARIES  OF  THE  NAVY  ACQUISITION 

AND  PROCUREMENT 

As  shown  in  Figure  5,  SYSCOM  commanders  such  as  NAVAIR  and  NAVSEA 
fall  under  ASN  (RDA);  similarly,  Deputy  Assistant  Secretaries  of  the  Navy  (Acquisition 
and  Procurement)  (DASN  [AP])  falls  under  ASN  (RDA).  DASN  (AP)  provides  policy 
and  procedures  for  the  Navy’s  world- wide  acquisition  system.  As  stated  on  the  website, 
DASN  (AP)’s  mission  is  “to  shape  acquisition  and  logistics  policies  that  assure  sailors 
and  marines  are  mission  capable,  and  have  a  technological  edge  over  adversaries.”  DASN 
(AP)  has  four  primary  goals,  to  advise  ASN  (RDA),  serve  as  DON  Competition 
Advocate  General,  establish  acquisition  and  logistics  policy,  and  improve  the  acquisition 
systems  (DASN  AP,  2015). 

D.  NAVSEA  ORGANIZATION 

NAVSEA  is  the  DON’s  primary  shipbuilding  agency.  According  to  NAVSEA’s 
official  website  (2014),  its  mission  is  to  “design,  build,  deliver,  and  maintain  ships  and 
systems  on  time  and  on  budget.”  Their  organization  consists  of  command  staff  and 
affiliated  PEOs  including:  Ships,  Littoral  Combat  Ships,  Submarines,  Carriers  and 
Integrated  Warfare  Systems  as  seen  in  Figure  6.  NAVSEA  manages  the  life-cycle  of  an 
acquisition  programs  from  planning  to  retirement.  Their  staff  consists  of  60,000  civilians, 
military  and  contractors.  Compared  to  other  five  SYSCOMs,  NAVSEA  has  the  highest 
yearly  budget  of  $30  billion.  NAVSEA  handles  150  acquisitions  programs,  and  oversees 
billions  of  dollars  in  foreign  military  sales  (NAVSEA,  2014).  NAVSEA  operates  in  16 
states  with  33  activities  and  plays  a  major  role  in  the  NAVY  Enterprise  that  is  responsible 
of  guiding  resource  sponsors  that  regulates  technical  standards  for  combat  systems 
(NAVSEA,  2014).  NAVSEA’s  affiliated  PEOs  are: 

•  PEO  for  Ships 

•  PEO  for  Littoral  Ships 

•  PEO  for  Submarines 

•  PEO  for  Integrated  Warfare  Systems 
(NAVSEA,  2014) 
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Figure  6.  NAVSEA  Corporate  Leadership  2014  (from  NAVSEA,  2014) 


As  per  NAVSEA,  they  locates  many  of  its  field  activities  in  different  parts  of  the 
country  in  order  to  assist  its  numerous  customers,  such  as  the  fleet  and  DOD,  providing 
products  and  support,  along  with  engineering,  scientific,  technical,  and  logistical 
expertise.  Figure  7  illustrates,  the  headquarters  of  NAVSEA  is  located  in  the  Washington, 
DC,  Navy  Yard.  NAVSEA’s  four  shipyards  include  Norfolk  Naval  Shipyard  in 
Portsmouth,  Virginia;  Pearl  Harbor  Naval  Shipyard  and  Intermediate  Maintenance 
Facility  (IMF)  in  Pearl  Harbor,  Hawaii;  the  Portsmouth  Naval  Shipyard  in  Kittery, 
Maine;  and  Puget  Sound  Naval  Shipyard  and  IMF  in  Bremerton,  Washington.  NAVSEA 
has  two  warfare  centers:  the  Naval  Surface  Warfare  Center  (NSWC)  and  the  Naval 
Undersea  Warfare  Center  (NUWC).  The  warfare  centers  play  an  important  role  in 
supplying  various  support  and  services  to  satisfy  customer  requirements.  The  support  and 
services  include  technical  operations,  people,  technology,  engineering,  and  products. 
They  are  considered  key  players  in  analyzing  and  assessing  the  various  ship  and 
submarine  systems.  These  include  combat  systems,  ordnance,  mines,  and  strategic 
systems  products  and  support  (NAVSEA,  2014).  Figure  7  illustrates  the  NAVSEA  field 
activities. 
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Figure  7.  NAVSEA  Field  Activities  (from  NAVSEA,  2014) 


SEA  02,  the  contracting  office  for  NAVSEA,  processes  almost  $24  billion  in 
contracts  every  year.  SEA  02  typically  awards  contracts  for  new  procurement  for 
construction;  repair  of  ships  and  submarines;  major  weapon  systems  and  other  types  of 
maintenance.  The  procurement  process  includes  developing  and  planning  acquisition 
strategies.  SEA  02  uses  different  procurement  methods  to  fulfill  the  warfighters  needs 
such  as:  solicitation,  negotiation  and  award  of  contracts;  as  well  as  administration  of 
contract  performance  (NAVSEA,  2014). 

NAVSEA  states  that  SEA  02  has  contracts  for  the  following  requirements: 

•  Ships,  shipboard  weapons,  and  combat  systems 

•  Design  and  integration 

•  Maintenance  and  repair 

•  Modernization  and  conversion 

•  Technical,  industrial  and  logistics  supports 

•  Other  professional  services,  such  as  engineering,  finance,  and  program 
management 

(NAVSEA,  2014) 
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As  shown  in  Figure  8,  SEA  02  is  has  five  divisions.  First,  SEA  022  is  the 
shipbuilding  division  that  deals  with  four  types  of  contracts  such  as  Cost-Plus  Fixed  Fee 
(CPFF),  Cost-Plus  Award  Fee  (CPAF),  and  Cost-Plus  Incentive  Fee  (CPIF)  contract 
types.  Second,  SEA  024  is  the  ship  repair  division  that  primarily  uses  CPFF  contracts. 
Third,  SEA  025  is  the  surface  systems  division  that  primarily  uses  Finn  Fixed  Priced 
(FFP)  two  types  of  contracts  such  as  Fixed-Plus  Incentive  Firm  (FPIF)  contracts.  Fourth, 
SEA  026  is  the  submarine  systems  division  that  procures  hardware  and  uses  FFP  and 
FPIF  contracts  (Graham,  Lewis,  &  Wallace,  2010). 
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Figure  8.  Organizational  Chart  of  Contract  Sea  02  (from  Graham  et  ah,  2010) 


E.  NAVAIR  ORGANIZATION 

Headquartered  in  Patuxent  River,  Maryland,  NAVAIR  was  established  in  1966 
and  is  the  primary  agency  responsible  for  the  development  and  procurement  of  Navy  and 
Marine  Corps  aviation  assets  (NAVAIR,  2014).  According  to  the  website,  NAV AIR’s 
mission  is  “to  provide  full  life-cycle  support  of  naval  aircraft,  weapons,  and  systems 
operated  by  sailors  and  marines”  into  eight  functional  areas,  they  are:  “research,  design, 
development  and  systems  engineering;  acquisition;  test  and  evaluation;  training  facilities 
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and  equipment;  repair  and  modification;  in-service  engineering  and  logistics  support” 
(NAVAIR,  2014). 

Structurally,  NAVAIR  is  also  organized  into  eight  communities  of  practice  (CoP) 
such  as:  “program  management,  contracts,  research  and  engineering,  test  and  evaluation, 
logistics  and  industrial  operations,  corporate  operations,  comptroller  and  counsel” 
(NAVAIR,  2014).  The  primary  recipients  of  NAVAIR’s  support  are  the  Program 
Executive  Officers  (PEOS)  and  the  program  managers  (PMs)  who  manage  the  acquisition 
“program’s  cost,  schedule,  and  performance  requirements.”  Typically  this  support  takes 
the  form  of  a  combination  of  the  following:  “people,  processes,  tools,  training,  mission 
facilities,  and  core  technologies”  (NAVAIR,  2014). 

As  per  NAVAIR,  there  are  four  affiliated  PEOs  are: 

•  PEO  for  Tactical  Aircraft  Programs:  PEO  (T) 

•  PEO  for  Air  Anti  Surface  Warfare  (ASW),  Assault  and  Special  Mission 
Programs:  PEO  (A) 

•  PEO  for  Unmanned  Aviation  and  Strike  Weapons:  PEO  (U&W) 

•  PEO  for  Joint  Strike  Fighter:  PEO  (JSF;  which  alternates  service  lead  with 
the  Air  Force) 

(NAVAIR  20 14) 

As  shown  in  Figure  9,  NAVAIR  has  eight  core  competencies:  Program 
Management  (AIR  1.0),  Contracts  (AIR  2.0),  Research  &  Engineering  (AIR  4.0),  Test  & 
Evaluation  (AIR  5.0),  Logistics  &  Industrial  Operations  (AIR  6.0),  Corporate  Operations 
(AIR  7.0),  and  the  Comptroller  (AIR  10.0)  (NAVAIR,  2014). 
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Figure  9.  NAVAIR  Organization  Chart  (from  NAVAIR,  2014) 


Figure  10  shows  NAVAIR’s  major  sites.  China  Lake  and  Point  Mugu  comprise 
the  weapons  division.  Lake  Hurst,  Cherry  Point,  and  Patuxent  River  comprise  the  aircraft 
division.  Finally,  Jacksonville  and  North  Island  are  the  depot/industrial  sites. 
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Figure  10.  NAVAIR  Major  Sites  (from  NAVAIR,  2014) 

NAVAIR  2.0  is  “accountable  for  contracting  supplies,  services,  and  material 
requirements  of  Integrated  Program  Teams  (IPT),  Program  Support  Teams  (PST),  and 
Enterprise  Teams”  (ET;  NAVAIR,  2014).  As  shown  in  Figure  11,  NAVAIR  2.0  has  six 
departments  (AIR  2.1,  AIR  2.2,  AIR  2.3,  AIR  2.4,  AIR  2.5,  and  AIR  2.6). 
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Figure  11.  NAVAIR  2.0  Organization  Chart  (from  NAY  AIR,  2014) 


F.  WHY  NAVAIR  AND  NAVSEA  FOR  THIS  RESEARCH? 

NAVAIR  and  NAVSEA,  two  major  SYSCOMs,  were  chosen  for  this  research 
project  because  they  have  the  largest  procurement  organizations  in  the  Navy  that  procure 
not  only  simple  goods  and  services  but  also  complex  systems.  These  two  commands 
conduct  multiple  contracting  source  selections,  which  consist  of  sufficient  mixture  of 
LPTA  and  the  Tradeoff  source  selection  strategies  to  answer  our  research  questions. 

G.  SUMMARY 

This  chapter  provided  an  overview  of  the  DOD  acquisition  organization  and 
primarily  focused  on  the  Navy’s  organization  of  acquisition  activities.  The  chapter  also 
included  a  discussion  of  the  two  major  systems  commands,  NAVAIR  and  NAVSEA 
specifically  their  organization,  mission,  and  contracting  divisions.  The  next  chapter 
discusses  how  data  was  accessed,  the  statistical  analysis  of  that  data,  the  findings  of  the 
analysis,  implication  and  results,  and  areas  for  further  research. 
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IV.  ANALYSIS 


In  this  chapter,  we  discuss  the  research  methodology  and  analysis  conducted  to 
answer  the  research  questions  presented  in  Chapter  I.  Specifically,  we  discuss  the  source 
of  the  data,  the  data  collection  method,  and  how  the  data  were  analyzed.  We  also  include 
a  description  of  the  spreadsheet  we  used  and  the  type  of  data  collected. 

A.  OVERVIEW  OF  THE  DATA 

The  purpose  of  this  research  is  to  explore  potential  relationship  between  the 
source  selection  strategy  (LPT A  or  Tradeoff)  and  resultant  contract  outcomes;  thus 
NAVAIR  and  NAVSEA  provided  us  the  best  option  within  the  Navy  to  collect  data  that 
captures  the  entire  contracting  management  process.  As  the  Navy’s  two  largest 
SYSCOMs,  NAVAIR  and  NAVSEA  have  a  combined  fiscal  year  2015  budget  of  $53 
billion  (NAVAIR  &  NAVSEA,  2014).  Given  their  wide  acquisition  authority,  they  are 
ideal  sources  of  data  for  our  research,  namely,  completed  contracts. 

1.  Data  Source 

To  collect  the  data  needed,  we  manually  reviewed  hard  copies  of  completed 
contracts  at  NAVAIR  and  NAVSEA’s  contract  file  repositories.  All  three  members  of  the 
research  team  traveled  to  NAVAIR  and  NAVSEA  and  spent  two  days  reviewing  contract 
files  at  each  location.  To  capture  the  data,  we  used  a  spreadsheet  developed  by  our 
advisors,  Professors  Rene  Rendon  and  Karen  Landale.  The  spreadsheet  was  designed  to 
capture  all  the  relevant  details  of  a  procurement  that  might  affect  contract  outcomes.  To 
maximize  our  efficiency,  we  sent  advance  copies  of  our  spreadsheet  to  each  command 
and  asked  for  assistance  locating  relevant  contract  files.  Both  commands  were  quite 
accommodating  to  our  request  for  access  to  their  contract  files,  and  each  provided  a 
workstation  and  a  representative  to  assist  us  with  locating  files. 

Upon  arriving  at  NAVAIR,  our  first  stop,  we  were  overwhelmed  by  the  volumes 
of  contracts  files  that  lined  the  shelves.  Our  initial  assessment  of  each  file  room  was  that 
we  would  have  no  trouble  collecting  a  large  sample  of  the  contract  data.  Unfortunately, 
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individual  contract  files  proved  to  be  much  larger  and  more  complex  than  we  had 
anticipated  and  far  more  difficult  to  mine  for  the  pertinent  data  fields.  While  each 
command  employs  the  use  of  a  contract  file  checklist  that  contained  the  elements  of  FAR 
4.803,  we  observed  that  about  a  third  of  the  contracts  we  reviewed  did  not  follow  it  as 
prescribed.  Some  contract  files  seemed  to  only  include  the  essential  documents  while 
others  tended  to  include  a  lot  of  extra  data  resulting  in  contract  files  spanning  multiple 
volumes.  We  found  that  the  condition  of  file  room  and  contract  files  at  NAVSEA  to  be 
quite  similar  to  those  at  NAVAIR. 

2.  Data  Description 

The  spreadsheet  we  used  to  collect  our  data  was  designed  to  capture  information 
from  all  six  steps  of  the  contract  management  process.  The  spreadsheet  is  divided  into 
five  overarching  categories  that  seek  to  provide  a  comprehensive  overview  of  each 
contract:  (1)  basic  contract  infonnation,  (2)  acquisition  complexity,  (3)  environmental 
factors,  (4)  outcomes  variables  and  (5)  other  relevant  contract  infonnation.  The  basic 
infonnation  section  captures  identifying  features  of  the  contract  such  as  contract  number, 
North  American  Industry  Classification  System  (NAICS)  code,  and  Product  or  Service 
Code  (PSC).  The  acquisition  complexity  section  captures  data  from  both  the  solicitation 
and  award  phases.  Some  of  the  high  points  of  this  section  include  contract  type,  whether 
the  contract  is  a  small  business  set  aside,  the  dollar  value  of  the  requirement,  award  fee 
criteria  and  incentives  (as  applicable).  The  environmental  factors  section  addresses  the 
evaluation  phase  and  focuses  primarily  on  the  actions  of  the  source  selection  team.  The 
outcome  variables  address  some  pre-award  and  post-award  factors  (e.g.,  they  include  the 
number  of  solicitation  amendments,  PALT  and  perfonnance  rating  data).  Finally,  the 
other  relevant  contract  section  addresses  contracting  officer  communique  (e.g.,  evaluation 
notices,  clarification  request  and  award  notices).  For  a  more  in-depth  look  at  the 
spreadsheet,  please  refer  to  the  appendix. 
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B.  DATA  ANALYSIS 

In  this  section,  we  describe  our  data,  provide  insight  into  our  dataset,  and  discuss 
the  methodology  used  to  analyze  the  data. 

1.  Data  Description 

For  this  analysis,  we  have  two  dependent,  or  outcome,  variables  (DVs): 
procurement  administrative  lead-time  (PALT)  and  Contractor  Performance  Assessment 
Rating  System  (CPARS)  data.  PALT  assesses  “time  to  contract”  by  calculating  the 
number  of  days  between  receipt  of  the  requisition  and  contract  award.  It  is  a  continuous 
variable.  CPARS  serves  as  a  proxy  measure  of  contract  success  or  failure  by  using  the 
ratings  given  to  each  contract  once  complete.  CPARS  ratings  are  given  in  Likert-style 
responses  where  l=Unsatisfactory,  2=Marginal,  3=Satisfactory,  4= Very  Good,  and 
5=Excellent.  In  this  case,  we  calculated  the  overall  CPARS  score  by  averaging  the 
following  CPARS  factors:  quality,  schedule,  management  of  key  personnel,  and  small 
business  use.  While  it  is  preferable  to  use  each  CPARS  rating  as  an  individual  aspect  of 
contract  success  (i.e.,  allow  each  CPARS  rating  to  be  a  measurable  contract  outcome), 
our  sample  size  was  not  large  enough,  nor  were  our  cases  complete  enough,  to  perform 
such  analyses.  Hence,  the  average  score  was  used. 

We  have  one  independent  variable,  or  IV.  Independent  variables  are  those  that  can 
be  manipulated  by  the  researcher  (or  user)  and  evoke  a  change  in  the  outcome,  or  DV. 
Our  IV  concerns  the  contract  methodology  used  for  the  contract:  LPTA  or  Tradeoff. 
Contracting  source  selection  strategy  is  a  choice  made  by  the  “user”  (the  integrated 
product  team,  which  includes  the  Contracting  Officer),  hence  it  is  considered  an  IV.  Our 
IV  is  labeled  LPTATO  and  it  is  a  binary  variable  where  0=LPTA  and  l=Tradeoff 

Finally,  we  have  one  covariate  variable.  Covariates  are  secondary  variables  that 
can  also  affect  the  relationship  of  primary  interest:  the  relationship  between  the  IV  and 
the  DV.  In  particular,  covariates  are  variables  other  than  the  IV  that  may  substantially 
affect  the  DV.  Our  covariate  is  contract  dollar  value  (VALUE)  and  it  is  a  continuous 
variable.  The  dollar  value  of  a  contract  affects  the  number  of  reviews  it  has  to  go  through, 
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thus  affecting  the  PALT.  Higher  dollar  contracts  typically  have  a  more  robust  review 
process,  and  thus  longer  PALTs.  The  opposite  is  typically  true  for  lower  dollar  value 
contracts.  In  this  case,  we  hope  to  parcel  out  the  effect  of  the  covariate  VALUE  in  order 
to  more  clearly  see  the  effect  the  contracting  methodology  (LPTATO)  has  on  the 
outcome  variables  (PALT  and  CPARS). 

3.  Descriptive  Statistics 

Basic  descriptive  statistics  for  each  variable  are  shown  in  Table  2. 2  The  table 
presents  three  figures  for  each  variable:  (1)  the  total  for  all  the  data,  (2)  the  total  for 
LPTA  contracts  and  (3)  the  total  for  Tradeoff  contracts. 


Table  2.  Data  Breakdown 


Descriptive  Statistics 

Variable 

Obs 

Mean 

SD 

Min 

Max 

PALT 

(days) 

36 

398.17 

265.80 

3 

953 

6 

170.67 

225.96 

3 

623 

30 

443.67 

252.21 

112 

953 

CPARS 

(rating) 

20 

4.04 

.97 

2 

5 

2 

3.13 

.18 

3 

3.25 

18 

4.14 

.97 

2 

5 

VALUE 

(dollars) 

38 

$65,300,000 

$105,000,000 

$238,410 

$450,000,000 

7 

$67,200,000 

$169,000,000 

$238,410 

$450,000,000 

31 

$64,800,000 

$88,800,000 

$1,199,776 

$353,000,000 

*Bold=total  for  all  data,  non-italicized=LPTA,  italicized=Tradeoff 


2  One  outlying  observation  was  deleted  from  the  dataset.  Analyses  were  performed  both  before  and 
after  deletion.  The  outlying  observation  did  not  affect  overall  significance  of  the  results,  however  because 
the  graphics  were  clearer  without  the  observation,  it  was  removed.  All  results  presented  in  this  paper 
exclude  the  outlying  observation. 
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4.  Data  Issues 


With  36  cases,  our  sample  size  is  somewhat  small.  Power  calculations  suggest  the 
need  for  14  cases  of  each  contracting  source  selection  strategy  (i.e.,  14  LPTA  cases  and 
14  Tradeoff  cases)  in  order  to  achieve  adequate  power  (a  =  .05,  (3  =  .80).  Our  data  is 
unbalanced  with  respect  to  the  number  of  cases  for  each  contracting  source  selection 
strategy.  There  are  6  LPTA  cases  and  30  Tradeoff  cases.  This  unbalanced  design  can 
cause  ambiguity  about  the  mean  as  the  intercept  and  make  assignment  of  sums  of  squares 
more  difficult.  There  are,  however,  solutions  to  these  issues.  A  weighted  mean  can  be 
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used  in  place  of  the  grand  mean  and  the  STATA  software  automatically  handles  the 
assignment  of  the  sums  of  squares.  Thus,  we  proceeded  with  our  analysis  despite  these 
issues. 


5.  Analysis 

Because  our  intent  is  to  analyze  differences  in  contract  outcomes  (PALT  and 
CPARS)  based  on  contracting  methodology  (LPTA  or  Tradeoff),  a  group  comparison 
statistical  methodology  is  necessary.  In  other  words,  the  contracting  source  selection 
strategies  are  divided  into  two  groups  (LPTA  and  Tradeoff),  and  we  seek  to  find  if  there 
are  differences  in  contract  outcomes  (PALT  and  CPARS)  by  group. 

We  initially  used  a  technique  called  multivariate  analysis  of  covariance,  or 
MANCOVA,  to  assess  group  differences.  The  results  showed  that  there  were  no 
differences  in  contract  outcomes  based  on  the  contracting  methodology  used.  Regardless 
of  whether  the  acquisition  team  chose  a  LPTA  or  Tradeoff  source  selection  strategy,  the 
lead-time  required  to  put  the  requirement  on  contract  (PALT)  and  the  success  of  the 
contract  (as  measured  by  CPARS  ratings)  did  not  vary  (i.e.,  were  not  significantly 
different).  We  suspected  that  the  results  may  be  different  if  we  examined  just  one 
contract  outcome  at  a  time,  thus  post-hoc  analyses  were  perfonned  in  which  PALT  and 
CPARS  were  analyzed  separately.  Because  we  were  now  assessing  the  outcome  variables 


^  The  grand  mean  would  be  the  intercept  in  a  balanced  design. 
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individually,  the  methodology  changed  from  a  multivariate  test  (MANCOVA)  to  a 
univariate  test,  known  simply  as  analysis  of  covariance,  or  ANCOVA. 

ANCOVA  addresses  the  following  questions: 

•  Are  mean  differences  among  the  groups  (after  adjusting  for  covariate 
effects)  likely  to  have  occurred  by  chance? 

•  Taken  from  another  angle,  is  there  a  significant  difference  between  the 
mean  value  for  PALT  in  the  LPTA  acquisitions  versus  the  mean  value  for 
PALT  in  the  Tradeoff  acquisitions  once  the  effect  of  the  covariate 
(VALUE)  has  been  parceled  out? 

ANCOVA  examines  the  relationships  between  the  dependent  variables  (PALT 
and  CPARS)  and  the  independent  variable  (choice  of  LPTA  or  Tradeoff  strategy)  while 
taking  into  account  the  effect  the  covariate  (VALUE)  might  have  on  the  outcome 
variables  (PALT  and  CPARS). 

6.  Assumption  Testing 

Before  conducting  the  ANCOVA,  certain  assumptions  about  the  data  were  tested. 
First,  we  assessed  univariate  nonnality  by  performing  the  Shapiro-Wilk  test.  Both  PALT 
and  contract  value  (VALUE)  were  deemed  to  be  non-nonnal.  PALT  was  normalized  via 
a  square  root  transformation,  while  a  logarithmic  transformation  was  performed  on 
VALUE.  Both  variables  passed  the  Shapiro-Wilk  nonnality  test  once  transformed. 

Second,  we  assessed  linearity  by  examining  scatter  plots  of  the  dependent 
variables  (PALT  and  CPARS)  and  the  covariate  variable  (VALUE).  The  plots  revealed 
fairly  linear  relationships  between  the  variables. 

Third,  we  assessed  homogeneity  of  regression  by  perfonning  an  analysis  of 
variance  (ANOVA,  which  is  practically  the  same  as  an  ANCOVA  but  does  not  contain  a 
covariate  variable)  that  included  the  independent  variable  (LPTA  or  Tradeoff),  the 
covariate  VALUE,  and  the  interaction  between  the  independent  variable  and  the 
covariate.  The  interaction  tenn  was  not  significant,  which  indicates  that  the  relationship 
between  the  dependent  variables  (PALT  and  CPARS)  and  covariate  (VALUE)  is  the 
same  at  both  levels  of  the  independent  variable  (LPTA  or  Tradeoff).  Hence,  the 
assumption  of  homogeneity  of  regression  is  upheld. 
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Finally,  we  checked  for  homogeneity  of  variance  between  groups  using  Bartlett’s 
Test.  The  results  showed  the  difference  in  variance  between  the  groups  (LPTA  and 
Tradeoff)  are  not  significant,  thus  the  assumption  of  homogeneity  of  variance  is  upheld. 


C.  RESULTS 

Using  CPARS  as  the  dependent  variable  produced  no  significant  findings. 
Essentially,  we  find  no  significant  differences  in  contract  success  (as  measured  by 
CPARS  data)  between  the  two  methodologies  (LPTA  and  Tradeoff).  Table  3  shows  the 
results  of  this  analysis. 


Table  3 .  ANCOVA  Using  CPARS  as  the  DV 


ANCOVA  Using  CPARS  as  the  DV 

Source 

Partial  SSA 

df 

MSAA 

F 

Prob  >  F 

Model 

3.66 

2 

1.83 

2.18 

.1433  ns 

VALUE 

1.81 

1 

1.81 

2.16 

.1601  ns 

LPTATO 

1.77 

1 

1.77 

2.11 

.1650  ns 

Residual 

14.25 

17 

.84 

Total 

17.91 

19 

.94 

A  Partial  sum  of  squares  AA  Mean  square 

Number  of  Observations  =  20 

Root  Mean  Squared  Error  =  .92 

R2  =  .2043 

Adjusted  R2  =  .  1 107 

Using  PALT  as  the  dependent  variable,  however,  showed  significant  differences 
exist  in  the  lead-time  for  acquisitions  that  use  LPTA  versus  acquisitions  that  use 
Tradeoff,  even  when  contract  dollar  value  is  taken  into  account.  Specifically,  LPTA 
acquisitions  are  much  shorter  than  Tradeoff  acquisitions  (Mean  LPTA  =  170.67  days, 
Mean  Tradeoff  =  443.67  days).  Table  4  provides  the  results  in  summary  fonnat. 
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Table  4.  ANCOVA  Using  PALT  as  the  DV 
ANCOVA  Using  PALT  as  the  DV 


Source 

Partial  SSA 

df 

MSAA 

F 

Prob  >  F 

Model 

435.24 

2 

217.62 

5.57 

.0083** 

VALUE 

12.24 

1 

12.24 

.31 

.5795  ns 

LPTATO 

197.27 

1 

197.27 

5.05 

.0315* 

Residual 

1290.03 

33 

39.09 

Total 

1725.26 

35 

49.29 

*p<.05  **p<.01 

A  Partial  sum  of  squares  AA  Mean  square 

Number  of  Observations  =  36 

Root  Mean  Squared  Error  =  6.25 

R2  =  .2523 

Adjusted  R2  =  .2070 

As  a  final  post-hoc  analysis,  we  examined  whether  there  were  significant 
differences  in  PALT  based  on  the  procuring  organization  (NAVSEA  vs.  NAVAIR).  No 
significant  differences  were  found. 

D.  DISCUSSION 

The  results  of  this  research  should  be  viewed  critically  given  the  limited  size  and 
the  unbalance  nature  of  the  sample.  Our  findings  clearly  suggest  that  PALT  is 
significantly  shorter  by  227.5  days  when  contracts  are  awarded  based  on  an  LPTA  source 
selection  strategy,  empirically  lending  support  to  what  was  previously  only  an  anecdotal 
belief.  This  result  was  found  even  when  contract  dollar  value  was  included  in  the  model. 
In  the  DOD,  the  dollar  value  of  the  acquisition  “trips”  certain  evaluation  and  review 
thresholds.  The  higher  the  dollar  value,  the  more  thresholds  the  requirement  must  pass 
prior  to  award,  thus  increasing  lead  time.  Finding  these  results  with  dollar  value  included 
in  the  model  lends  credence  to  the  notion  that  LPTA  acquisitions  are  in  fact  shorter  than 
Tradeoff  source  acquisitions.  Given  the  dynamic  nature  of  the  DOD,  shorter  acquisition 
lead-time  is  typically  preferred  because  it  directly  translates  to  delivering  the  requirement 
to  the  warfighter  sooner. 
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E.  SUMMARY 

This  chapter  provided  the  results  of  our  research.  The  chapter  began  with  an 
overview  of  the  data,  to  include  a  description  of  the  data  sources  and  a  description  of  the 
spreadsheet  created  to  gather  the  data.  Next,  the  data  and  methodology  were  described, 
and  the  results  of  the  research  were  presented.  The  final  chapter  includes  a  summary  of 
the  research,  conclusions  and  areas  for  further  research. 
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V.  SUMMARY,  CONCLUSION,  AND  AREAS  FOR  FURTHER 

RESEARCH 


This  chapter  provides  a  summary  of  the  research,  provides  answers  to  our 
research  questions,  and  recommends  areas  of  further  research. 

A.  SUMMARY 

The  DOD  spends  billions  of  dollars  annually  acquiring  weapons  systems,  supplies 
and  services  to  support  the  needs  of  the  warfighters.  Government  acquisition 
professionals  use  the  six  steps  of  the  contract  management  process  to  award  the  contracts 
for  these  good  and  services.  The  contract  management  process  consists  of  pre-award, 
award,  and  post  award  phases.  A  key  step  in  the  contract  management  process  is  source 
selection,  which  occurs  during  the  award  phase.  The  two  primary  source  selection 
strategies  used  to  obtain  the  best  value  for  the  government  are  LPTA  and  Tradeoff.  The 
purpose  of  this  research  is  to  detennine  potential  relationships  between  the  source 
selection  strategy  (LPTA  or  Tradeoff)  and  resultant  contract  outcomes.  Data  were 
collected  from  completed  contract  files  at  NAVAIR  and  NAVSEA,  paying  particularly 
attention  to  the  source  selection  strategies  chosen  for  each  contract.  The  data  were  then 
analyzed  to  determine  how  the  choice  of  source  selection  strategy  affects  contract 
outcomes. 

B.  CONCLUSION 

To  conclude  this  research,  we  present  answers  to  the  research  questions  posed  in 
Chapter  I. 

(1)  How  does  the  source  selection  strategy  affect  pre-award  metrics  (e.g., 
PALT,  number  of  solicitation  amendments,  number  of  protests)? 

Our  analyses  proved  that  significant  differences  exist  in  the  PALT  based  on 
source  selection  strategy.  As  illustrated  in  Figure  12,  the  PALT  mean  time  for  LPTA  is 
170.67  days  versus  443.67  days  for  Tradeoff.  The  results  of  our  research  lend  support  to 
the  anecdotal  assumption  that  Tradeoff  contracts  have  a  longer  lead-time  for  award. 
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Given  the  unbalanced  nature  of  the  data  collected  (30  Tradeoffs  and  6  LPTA)  it  appears 
that  major  weapons  system  acquisition  commands  seems  to  favor  a  Tradeoff  source 
selection  strategy,  as  one  would  expect  given  the  developmental  nature  of  many  weapon 
systems. 


Source  Selection  Strategy 


Figure  12.  LPTA  versus  Tradeoff  in  Days 


The  results  of  the  analyses  did  not  yield  any  significant  findings  regarding  the 
number  of  solicitation  amendments  or  the  number  of  protests  based  on  the  source 
selection  strategy  chosen.  It  also  did  not  produce  any  significant  findings  that  would  lead 
to  a  conclusion  that  a  particular  source  selection  strategy  (LPTA  or  Tradeoff)  produces  a 
higher  or  lower  occurrence  of  amendments  or  protests.  Of  the  36  cases  evaluated,  less 
than  10%  (3  of  36  contracts)  received  a  protest.  Each  protest  was  made  to  the  GAO,  two 
were  dismissed  and  one  denied.  Given  these  few  cases  no  further  analysis  was  conducted. 

(2)  Flow  does  the  source  selection  strategy  affect  post-award  outcomes  (e.g., 
Contractor  Performance  Assessment  Reporting  Systems  [CPARS]  ratings, 
Earned  Value  Management  [EVM]  performance  metrics)? 

Only  55%  of  the  contract  data  we  collected  had  CPARS  ratings.  That  is  not  to  say 
that  CPARS  was  not  performed  on  these  contracts,  but  simply  that  our  PPIRs  search  did 
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not  produce  any  CPARS  data  for  45%  of  the  contracts  reviewed.  In  our  analysis  of  the  20 
contracts  with  CPARS  data,  we  found  no  significant  difference  in  contract  outcomes 
based  on  source  selection  strategy  (LPT A  or  Tradeoff).  However,  given  the  small  size  of 
our  sample,  the  reliability  of  this  finding  is  not  strong.  A  much  larger  sample  is  required 
to  accurately  assess  differences. 

We  were  unable  to  collect  EVM  perfonnance  metrics  data  on  any  of  the  contracts 
in  our  sample.  EVM  data  was  not  available  in  the  files  we  reviewed  and  our  attempts  to 
gain  access  to  Defense  Acquisition  Management  Information  Retrieval  (DAMIR)  failed. 
Therefore,  our  research  is  inconclusive  with  regard  to  EVM  perfonnance  differences 
based  on  source  selection  strategy. 

(3)  Does  one  source  selection  strategy  consistently  fare  better  than  the  other  in 
tenns  of  both  pre-award  metrics  and  post-award  outcomes? 

The  results  of  our  research  seem  to  suggest  that  an  LPTA  source  selection 
strategy  has  a  significantly  shorter  lead-time  to  contract  award.  That  said,  our  finding 
should  be  viewed  with  caution  as  our  sample  size  consisted  of  only  a  few  LPTA  cases 
(six) — a  sample  size  that  is  too  small  and  too  unbalanced  to  produce  fully  reliable  results. 
Also,  the  data  were  inconclusive  in  terms  of  linking  a  pre-award  metric  (i.e.,  number  of 
solicitation  modifications)  to  contract  outcomes. 

(4)  Do  the  contract  outcomes  (e.g.,  past  performance  data)  justify  the 
government  paying  a  premium  to  award  to  other  than  the  lowest  bidder? 

More  data  is  needed  to  answer  this  question.  In  particular,  more  CPARS  ratings 
and  EVM  perfonnance  metrics  are  required  to  detennine  if  the  premium  paid  to  award  a 
contract  to  the  other  than  lowest  bidder  is  justified. 

C.  AREAS  FOR  FURTHER  RESEARCH 

The  task  of  understanding  the  impact  of  a  source  selection  strategy  on  resultant 
contract  outcomes  is  a  topic  rich  for  further  research.  Our  research  is  valuable  in  that  it 
highlights  the  fact  that  more  research  is  required  to  better  understand  the  effects  of  source 
selection  strategy  on  contract  outcomes.  Our  recommendations  for  further  research  are 
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centered  on  the  areas  that  proved  to  be  the  most  difficult  for  us:  collecting  a  large,  well- 
balance  sample  size  and  obtaining  performance  data  (as  measured  by  CPARS  and  EVM). 

(1)  Larger  and  more  balanced  sample  size 

While  it  is  well  known  that  a  larger  sample  size  produces  more  reliable  results,  it 
is  important  to  note  that  collecting  data  proved  to  be  far  more  time  consuming  than 
anticipated.  Choosing  to  collect  data  from  NAVAIR  and  NAVSEA  had  both  advantages 
and  disadvantages.  The  advantage  is  that  as  SYSCOMs,  they  had  a  large  population  of 
complex  contracts  to  choose  from.  The  disadvantage  is  that  because  the  contracts  are 
typically  very  large  and  complex,  they  took  a  considerable  amount  of  time  to  mine  for 
data.  Therefore,  any  researcher  attempting  to  understand  this  topic  must  plan 
accordingly — scrubbing  a  large,  balanced  (equal  number  of  LPTA  and  Tradeoff 
contracts)  sample  will  required  a  significant  time  commitment.  Our  recommendation  is 
that  this  process  be  repeated  in  its  entirety  with  a  greater  emphasis  on  collecting  a  large 
and  balanced  sample. 

(2)  Greater  access  to  past  performance  data  (CPARS  and  EVM) 

We  had  very  limited  access  to  CPARS  and  no  access  to  EVM  at  all.  Therefore, 
we  were  unable  to  produce  any  reliable  findings  on  the  impact  of  a  particular  source 
selection  strategy  on  contract  outcomes.  Though  the  results  of  the  ANCOVA  analysis 
described  in  Chapter  IV  suggest  that  there  are  no  differences  in  outcomes  (as  measured 
by  CPARS  and  EVM)  between  the  two  source  selection  strategies  (LPTA  and  Tradeoff), 
we  place  little  confidence  in  this  finding  given  the  size  of  our  sample.  Of  the  36  cases,  we 
were  only  able  to  collect  CPARS  data  on  20  and  zero  data  on  EVM.  Further  research 
should  be  conducted  with  the  focus  of  evaluating  perfonnance  (CPARS  and  EVM)  to 
examine  contact  outcome.  Collecting  EVM  data  will  most  likely  require  future 
researchers  to  concentrate  on  major  acquisition  commands,  as  they  have  a  higher 
probability  of  having  contracts  that  use  EVM  performance  metrics. 
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APPENDIX.  DATA  COLLECTION  CATEGORIES 


Data  Collection  Catagories 

Basic  Info 

Acquisition  Complexity 

Observation 

# 

Predominant 

Contract 

Type 

#  Evaluation 

Factors 

Eval  Factor 

#4 

Life  Cycle 
Stage 

# 

Competitive 

Range 

Incentive  Fee 

Criterion  #5 

Service 

SB  Set  Aside 

Eval  Factor 

#1 

#  Subfactors 

in  EF#4 

Joint  Urgent 
Operational 
Need  (JUON) 

Single  or 
Multiple 
Award 

Award  Fee 

Criterion  #1 

Organization 

Consolidated 

Requirement 

#  Subfactors 

in  EF#1 

Eval  Factor 

#5 

#  CLINs 

On-Ramp 

Provision 

Award  Fee 

Criterion  #2 

Contract 

Number 

Multiyear 

Eval  Factor 

#2 

#  Subfactors 

in  EF#5 

Pre- 

Solicitation 
Conference/ 
Industry  Day 

Incentive  Fee 

Criterion  #1 

Award  Fee 

Criterion  #3 

Name 

#  Years 

#Subfactors 

in  EF#2 

# 

Subelements 

Pre-Proposal 

Conference 

Incentive  Fee 

Criterion  #2 

Award  Fee 

Criterion  #4 

NAICS 

Dollar  Value 

Eval  Factor 

#3 

ACAT/SCAT 

Level 

Pre- 

Award/Pre- 

Performance 

Conference 

Incentive  Fee 

Criterion  #3 

Award  Fee 

Criterion  #5 

PSC 

Trade  Off  or 

LPTA 

#  Subfactors 

in  EF#3 

FAR  Part 

#  Offerors 

Incentive  Fee 

Criterion  #4 
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Data  Collection  Catagories 


Environmental  Factor 

Outcome  Variables 

Other 

ft  Reviews 

#Team  Locations 

PALT- 

Requirement 
Receipt  Date 

Protest 

EVM  CV 

RFP 

Amendment 

Reason  2 

#  Evaluation 

Notices 

Review  1  Date 

ACE  Support 

PALT  -  RFP 

Issue  Date 

Protest  Level 

EVM  SV 

RFP 

Amendment 

Reason  3 

ft  ENs  Pre- 
Competitive 
Range 

Review  2  Date 

MIRT  Support 

PALT- 

Proposals 

Received 

Date 

Protest 

Results 

EVM  CPI 

RFP 

Amendment 

Reason  4 

ft  ENs  Post- 
Competitive 
Range 

Review  3  Date 

Use  of 

Collaboration 

Tools 

PALT- 

Contract 

Award  Date 

Corrective 

Action 

(Describe) 

EVM  SPI 

RFP 

Amendment 

Reason  5 

ft  Rounds  of 

ENs 

Review  4  Date 

Abundance/  Lack 
of  File 

Documentation 

Rating 

PALT  -  Days 

CPAR  Cost 

Nunn- 

McCurdy 

Breach 

RFP 

Amendment 

Reason  6 

ft 

Clarification 

Requests 

ft  Personnel  on 

Source  Selection 

Team 

State  of  the 
Economy 

Early 

Strategy  and 
Issues 

Session 
(ESIS)  Date 
(>$100M) 

CPAR  Quality 

# of RFP 

Amendment 

s 

Earned 

Award  Fee  % 

ft 

Deficiency 

Reports 

ASP  Date 

CPAR 

Business 

Relationship 

RFP 

Amendment 

Reason  1 

Earned 

Incentive 

Amount/ 

Available 

Basis  for 

Award 
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